In [1]:
!nvidia-smi
Tue Aug  9 12:52:02 2022       
+-----------------------------------------------------------------------------+
| NVIDIA-SMI 460.32.03    Driver Version: 460.32.03    CUDA Version: 11.2     |
|-------------------------------+----------------------+----------------------+
| GPU  Name        Persistence-M| Bus-Id        Disp.A | Volatile Uncorr. ECC |
| Fan  Temp  Perf  Pwr:Usage/Cap|         Memory-Usage | GPU-Util  Compute M. |
|                               |                      |               MIG M. |
|===============================+======================+======================|
|   0  Tesla P100-PCIE...  Off  | 00000000:00:04.0 Off |                    0 |
| N/A   36C    P0    26W / 250W |      0MiB / 16280MiB |      0%      Default |
|                               |                      |                  N/A |
+-------------------------------+----------------------+----------------------+
                                                                               
+-----------------------------------------------------------------------------+
| Processes:                                                                  |
|  GPU   GI   CI        PID   Type   Process name                  GPU Memory |
|        ID   ID                                                   Usage      |
|=============================================================================|
|  No running processes found                                                 |
+-----------------------------------------------------------------------------+
In [2]:
from google.colab import drive
drive.mount('/content/gdrive')
Mounted at /content/gdrive
In [3]:
%cd /content/gdrive/MyDrive/Atelier_Google_Colab/Projet_8

# List files to make sure we're in the expected directory.
!ls
/content/gdrive/MyDrive/Atelier_Google_Colab/Projet_8
data_redim  P08_01_notebook_modelisation.ipynb	saved  src
In [4]:
#!pip uninstall imgaug
#!pip install git+https://github.com/aleju/imgaug.git
Found existing installation: imgaug 0.4.0
Uninstalling imgaug-0.4.0:
  Would remove:
    /usr/local/lib/python3.7/dist-packages/imgaug-0.4.0.dist-info/*
    /usr/local/lib/python3.7/dist-packages/imgaug/*
Proceed (y/n)? y
  Successfully uninstalled imgaug-0.4.0

Looking in indexes: https://pypi.org/simple, https://us-python.pkg.dev/colab-wheels/public/simple/
Collecting git+https://github.com/aleju/imgaug.git
  Cloning https://github.com/aleju/imgaug.git to /tmp/pip-req-build-vq9g2i1k
  Running command git clone -q https://github.com/aleju/imgaug.git /tmp/pip-req-build-vq9g2i1k
Requirement already satisfied: six in /usr/local/lib/python3.7/dist-packages (from imgaug==0.4.0) (1.15.0)
Requirement already satisfied: numpy>=1.15 in /usr/local/lib/python3.7/dist-packages (from imgaug==0.4.0) (1.21.6)
Requirement already satisfied: scipy in /usr/local/lib/python3.7/dist-packages (from imgaug==0.4.0) (1.7.3)
Requirement already satisfied: Pillow in /usr/local/lib/python3.7/dist-packages (from imgaug==0.4.0) (7.1.2)
Requirement already satisfied: matplotlib in /usr/local/lib/python3.7/dist-packages (from imgaug==0.4.0) (3.2.2)
Requirement already satisfied: scikit-image>=0.14.2 in /usr/local/lib/python3.7/dist-packages (from imgaug==0.4.0) (0.18.3)
Requirement already satisfied: opencv-python in /usr/local/lib/python3.7/dist-packages (from imgaug==0.4.0) (4.6.0.66)
Requirement already satisfied: Shapely in /usr/local/lib/python3.7/dist-packages (from imgaug==0.4.0) (1.8.2)
Requirement already satisfied: imageio in /usr/local/lib/python3.7/dist-packages (from imgaug==0.4.0) (2.9.0)
Requirement already satisfied: tifffile>=2019.7.26 in /usr/local/lib/python3.7/dist-packages (from scikit-image>=0.14.2->imgaug==0.4.0) (2021.11.2)
Requirement already satisfied: networkx>=2.0 in /usr/local/lib/python3.7/dist-packages (from scikit-image>=0.14.2->imgaug==0.4.0) (2.6.3)
Requirement already satisfied: PyWavelets>=1.1.1 in /usr/local/lib/python3.7/dist-packages (from scikit-image>=0.14.2->imgaug==0.4.0) (1.3.0)
Requirement already satisfied: kiwisolver>=1.0.1 in /usr/local/lib/python3.7/dist-packages (from matplotlib->imgaug==0.4.0) (1.4.4)
Requirement already satisfied: cycler>=0.10 in /usr/local/lib/python3.7/dist-packages (from matplotlib->imgaug==0.4.0) (0.11.0)
Requirement already satisfied: pyparsing!=2.0.4,!=2.1.2,!=2.1.6,>=2.0.1 in /usr/local/lib/python3.7/dist-packages (from matplotlib->imgaug==0.4.0) (3.0.9)
Requirement already satisfied: python-dateutil>=2.1 in /usr/local/lib/python3.7/dist-packages (from matplotlib->imgaug==0.4.0) (2.8.2)
Requirement already satisfied: typing-extensions in /usr/local/lib/python3.7/dist-packages (from kiwisolver>=1.0.1->matplotlib->imgaug==0.4.0) (4.1.1)
Building wheels for collected packages: imgaug
  Building wheel for imgaug (setup.py) ... done
  Created wheel for imgaug: filename=imgaug-0.4.0-py3-none-any.whl size=971122 sha256=06bb79d041875cd32e55c4cc94a443f33613719bf73654539fe2919a3bc92e0b
  Stored in directory: /tmp/pip-ephem-wheel-cache-o3p89m3x/wheels/0c/78/b5/9303fae9d5e03df1f319adfe4e6534180b5c3232de11bc9a2f
Successfully built imgaug
Installing collected packages: imgaug
Successfully installed imgaug-0.4.0

Imports¶

In [1]:
%reload_ext autoreload
%autoreload 2
%matplotlib inline

import os
import random
import cv2
import time

import pandas as pd
import numpy as np

import matplotlib.pyplot as plt
import tensorflow as tf
from tensorflow.python.keras.callbacks import ModelCheckpoint, EarlyStopping
from tensorflow.keras.optimizers import Adam
from tensorflow.keras.metrics import MeanIoU, OneHotMeanIoU

from keras.models import load_model

import imgaug as ia
import imgaug.augmenters as iaa

# Personal libraries
import src.datagenerator as dg
import src.models as models
import src.helpers as helpers

Constantes¶

In [2]:
# Images
leftImg8bit_path = 'data_redim/leftImg8bit/'
# Masques
gtFine_path = 'data_redim/gtFine/'

checkpoint_path = 'saved/checkpoint/'
models_path = 'saved/models/'
history_path = 'saved/history/'

if not os.path.isdir('saved'):
  os.makedirs(checkpoint_path)
  os.makedirs(models_path)
  os.makedirs(history_path)

DIM_X = 256
DIM_Y = 256
INPUT_SHAPE = (DIM_X,DIM_Y,3)
NB_CLASSES = 8

monitor_val = 'val_mean_iou'
learning_rate = 1e-5
nbr_epochs = 50
patience = 14

Récupérer les chemins d'accès¶

In [3]:
# Recupère les chemins d'accès des fichiers dans le dossier précisé
def getPathFiles(dir_path, pattern = ""):

    train_path = dir_path + 'train/'
    test_path = dir_path + 'test/'
    val_path = dir_path + 'val/'

    train_path_files = []
    test_path_files = []
    val_path_files = []

    # Train set
    for dir in os.listdir(train_path):
        for file in os.listdir(train_path + dir):
            if pattern in file:
                train_path_files.append(train_path + dir + '/' +file)

    # Test set
    for dir in os.listdir(test_path):
        for file in os.listdir(test_path + dir):
            if pattern in file:
                test_path_files.append(test_path + dir + '/' +file)

    # Val set
    for dir in os.listdir(val_path):
        for file in os.listdir(val_path + dir):
            if pattern in file:
                val_path_files.append(val_path + dir + '/' +file)

    return train_path_files, test_path_files, val_path_files
In [4]:
# Listes des chemins des images
x_train_path, x_test_path, x_val_path = getPathFiles(leftImg8bit_path)
# Listes des chemins des masques
y_train_path, y_test_path, y_val_path = getPathFiles(gtFine_path, "labelIds.png")

# Echantillon préparé pour les tests
X_test, y_test = helpers.get_data_prepared(x_test_path[0:10], y_test_path[0:10], (DIM_X,DIM_Y))

Redimensionner les images pour le stockage dans le Cloud¶

In [ ]:
data_redim_path = 'data_redim'

list_dir = os.listdir()

if data_redim_path not in list_dir:
    os.mkdir('data_redim')

    for dir_1 in os.listdir('data'):
        if dir_1 != 'img':
            os.mkdir(data_redim_path + '/' + dir_1)
            for dir_2 in os.listdir('data' + '/' + dir_1):
                os.mkdir(data_redim_path + '/' + dir_1 + '/' + dir_2)
                for dir_3 in os.listdir('data' + '/' + dir_1 + '/' + dir_2):
                    os.mkdir(data_redim_path + '/' + dir_1 + '/' + dir_2 + '/' + dir_3)


    all_images_path = x_train_path + x_test_path + x_val_path + y_train_path + y_test_path + y_val_path

    for path in all_images_path:
        img = cv2.resize(cv2.imread(path), (1024, 512))
        new_path = path.replace('data', 'data_redim')
        cv2.imwrite(new_path, img)

else:
  print(f'Dossier {data_redim_path} déjà créé')
Dossier data_redim déjà créé

Exemple data augmentation¶

In [5]:
X = np.array([cv2.resize(cv2.imread(path_X), (DIM_X, DIM_Y)) for path_X in x_train_path[0:5]])
y = np.array([cv2.resize(helpers.convert_mask(cv2.imread(path_y,0)), (DIM_X, DIM_Y)) for path_y in y_train_path[0:5]])

X_augmented, y_augmented = helpers.augment_data(X, y)
y_augmented_arg = np.argmax(y_augmented, axis=3)
In [6]:
for i in range(len(X)):
    plt.subplots(1, 4, figsize=(21,5))

    plt.subplot(1, 4, 1)
    plt.imshow(X[i])

    plt.subplot(1, 4, 2)
    plt.imshow(X_augmented[i])

    plt.subplot(1, 4, 3)
    plt.imshow(y_augmented_arg[i])

    plt.subplot(1, 4, 4)
    plt.imshow(X_augmented[i])
    plt.imshow(y_augmented_arg[i], alpha=0.3)

    plt.show()

Data Generator¶

In [19]:
data_train = dg.DataGenerator(x_train_path, y_train_path, batch_size=20, dim=(DIM_X, DIM_Y), augmentation = False)
data_val = dg.DataGenerator(x_val_path, y_val_path, batch_size=20, dim=(DIM_X, DIM_Y), augmentation = False)

data_train_augmented = dg.DataGenerator(x_train_path, y_train_path, batch_size=10, dim=(DIM_X, DIM_Y), augmentation = True)
data_val_augmented = dg.DataGenerator(x_val_path, y_val_path, batch_size=10, dim=(DIM_X, DIM_Y), augmentation = True)

Modèles¶

Fonctions train & predict génériques¶

In [20]:
import json

synthese = {}
train_times = {}
if os.path.exists(models_path + 'train_times.json'):
    with open(models_path + 'train_times.json', 'r') as f:
        train_times = json.load(f)
In [21]:
def train_model(model, model_name, loss_function, augmented=False):
  filepath_check = checkpoint_path + model_name + '/'
  model_path = models_path + model_name
  history_path_file = history_path + model_name + '_history.json'

  if os.path.exists(model_path):
    if loss_function == "categorical_crossentropy":
      model = load_model(model_path)
    else:
      model = load_model(model_path, custom_objects={'dice_loss': dice_loss,'total_loss': total_loss}, compile=True)
    print(f'Récupération du modèle {model.name}')

    training_time = train_times[model_name]

    with open(history_path_file, 'r') as f:
      history = json.load(f)
    hist_df = pd.DataFrame.from_dict(history)

  else:
    start_train = time.time()

    # compile
    model.compile(optimizer=Adam(learning_rate=learning_rate), loss=loss_function, metrics=[OneHotMeanIoU(num_classes=NB_CLASSES, name='mean_iou')])

    mc = ModelCheckpoint(mode='max', filepath=filepath_check, monitor=monitor_val, save_best_only='True', save_weights_only='True')
    es = EarlyStopping(mode='max', monitor=monitor_val, patience=patience, verbose=1)

    callbacks = [mc, es]

    # fit
    if augmented:
      history = model.fit(data_train_augmented, epochs=nbr_epochs, validation_data=data_val_augmented, use_multiprocessing=True, workers=2, callbacks=callbacks)
    else:
      history = model.fit(data_train, epochs=nbr_epochs, validation_data=data_val, use_multiprocessing=True, workers=2, callbacks=callbacks)

    # convert the history.history dict to a pandas DataFrame:     
    hist_df = pd.DataFrame(history.history)

    # save history
    with open(history_path_file, mode='w') as f:
        hist_df.to_json(f)

    # Save the model
    model.load_weights(filepath_check)
    model.save(model_path)

    end_train = time.time()
    training_time = end_train - start_train
    train_times[model_name] = training_time

    with open(models_path + 'train_times.json', 'w') as f:
      json.dump(train_times, f)
  
  return model, hist_df, training_time
In [22]:
def predict_model(model, model_name, loss_type, training_time, show_pred=False):
  loss_score, mean_iou_score = model.evaluate(data_val)
  print("Pour le meilleur modèle on obtient :")
  print("mean_iou :", mean_iou_score)
  print("loss :", loss_score)

  start_pred = time.time()

  y_pred=model.predict(X_test)
  y_pred_argmax=np.argmax(y_pred, axis=3)

  end_pred = time.time()
  predict_time = end_pred - start_pred

  if show_pred:
    for index in range(10):
        print(index)
        plt.imshow(X_test[index])
        plt.imshow(y_pred_argmax[index],alpha = 0.4)
        plt.show()

  synthese[model_name] = helpers.add_model_in_synthese(loss_type, loss_score, mean_iou_score, training_time, predict_time)

  return y_pred_argmax

Modèle Unet de base¶

  • Métrique Mean_iou
  • Fonction loss 'categorical_crossentropy"
  • Pas d'augmentation de données
In [ ]:
# Build model
model = models.build_unet(INPUT_SHAPE, NB_CLASSES)
model_name = model.name + '_base'
model.summary()
softmax
Model: "U-Net"
__________________________________________________________________________________________________
 Layer (type)                   Output Shape         Param #     Connected to                     
==================================================================================================
 input_1 (InputLayer)           [(None, 256, 256, 3  0           []                               
                                )]                                                                
                                                                                                  
 conv2d (Conv2D)                (None, 256, 256, 64  1792        ['input_1[0][0]']                
                                )                                                                 
                                                                                                  
 batch_normalization (BatchNorm  (None, 256, 256, 64  256        ['conv2d[0][0]']                 
 alization)                     )                                                                 
                                                                                                  
 activation (Activation)        (None, 256, 256, 64  0           ['batch_normalization[0][0]']    
                                )                                                                 
                                                                                                  
 conv2d_1 (Conv2D)              (None, 256, 256, 64  36928       ['activation[0][0]']             
                                )                                                                 
                                                                                                  
 batch_normalization_1 (BatchNo  (None, 256, 256, 64  256        ['conv2d_1[0][0]']               
 rmalization)                   )                                                                 
                                                                                                  
 activation_1 (Activation)      (None, 256, 256, 64  0           ['batch_normalization_1[0][0]']  
                                )                                                                 
                                                                                                  
 max_pooling2d (MaxPooling2D)   (None, 128, 128, 64  0           ['activation_1[0][0]']           
                                )                                                                 
                                                                                                  
 conv2d_2 (Conv2D)              (None, 128, 128, 12  73856       ['max_pooling2d[0][0]']          
                                8)                                                                
                                                                                                  
 batch_normalization_2 (BatchNo  (None, 128, 128, 12  512        ['conv2d_2[0][0]']               
 rmalization)                   8)                                                                
                                                                                                  
 activation_2 (Activation)      (None, 128, 128, 12  0           ['batch_normalization_2[0][0]']  
                                8)                                                                
                                                                                                  
 conv2d_3 (Conv2D)              (None, 128, 128, 12  147584      ['activation_2[0][0]']           
                                8)                                                                
                                                                                                  
 batch_normalization_3 (BatchNo  (None, 128, 128, 12  512        ['conv2d_3[0][0]']               
 rmalization)                   8)                                                                
                                                                                                  
 activation_3 (Activation)      (None, 128, 128, 12  0           ['batch_normalization_3[0][0]']  
                                8)                                                                
                                                                                                  
 max_pooling2d_1 (MaxPooling2D)  (None, 64, 64, 128)  0          ['activation_3[0][0]']           
                                                                                                  
 conv2d_4 (Conv2D)              (None, 64, 64, 256)  295168      ['max_pooling2d_1[0][0]']        
                                                                                                  
 batch_normalization_4 (BatchNo  (None, 64, 64, 256)  1024       ['conv2d_4[0][0]']               
 rmalization)                                                                                     
                                                                                                  
 activation_4 (Activation)      (None, 64, 64, 256)  0           ['batch_normalization_4[0][0]']  
                                                                                                  
 conv2d_5 (Conv2D)              (None, 64, 64, 256)  590080      ['activation_4[0][0]']           
                                                                                                  
 batch_normalization_5 (BatchNo  (None, 64, 64, 256)  1024       ['conv2d_5[0][0]']               
 rmalization)                                                                                     
                                                                                                  
 activation_5 (Activation)      (None, 64, 64, 256)  0           ['batch_normalization_5[0][0]']  
                                                                                                  
 max_pooling2d_2 (MaxPooling2D)  (None, 32, 32, 256)  0          ['activation_5[0][0]']           
                                                                                                  
 conv2d_6 (Conv2D)              (None, 32, 32, 512)  1180160     ['max_pooling2d_2[0][0]']        
                                                                                                  
 batch_normalization_6 (BatchNo  (None, 32, 32, 512)  2048       ['conv2d_6[0][0]']               
 rmalization)                                                                                     
                                                                                                  
 activation_6 (Activation)      (None, 32, 32, 512)  0           ['batch_normalization_6[0][0]']  
                                                                                                  
 conv2d_7 (Conv2D)              (None, 32, 32, 512)  2359808     ['activation_6[0][0]']           
                                                                                                  
 batch_normalization_7 (BatchNo  (None, 32, 32, 512)  2048       ['conv2d_7[0][0]']               
 rmalization)                                                                                     
                                                                                                  
 activation_7 (Activation)      (None, 32, 32, 512)  0           ['batch_normalization_7[0][0]']  
                                                                                                  
 max_pooling2d_3 (MaxPooling2D)  (None, 16, 16, 512)  0          ['activation_7[0][0]']           
                                                                                                  
 conv2d_8 (Conv2D)              (None, 16, 16, 1024  4719616     ['max_pooling2d_3[0][0]']        
                                )                                                                 
                                                                                                  
 batch_normalization_8 (BatchNo  (None, 16, 16, 1024  4096       ['conv2d_8[0][0]']               
 rmalization)                   )                                                                 
                                                                                                  
 activation_8 (Activation)      (None, 16, 16, 1024  0           ['batch_normalization_8[0][0]']  
                                )                                                                 
                                                                                                  
 conv2d_9 (Conv2D)              (None, 16, 16, 1024  9438208     ['activation_8[0][0]']           
                                )                                                                 
                                                                                                  
 batch_normalization_9 (BatchNo  (None, 16, 16, 1024  4096       ['conv2d_9[0][0]']               
 rmalization)                   )                                                                 
                                                                                                  
 activation_9 (Activation)      (None, 16, 16, 1024  0           ['batch_normalization_9[0][0]']  
                                )                                                                 
                                                                                                  
 conv2d_transpose (Conv2DTransp  (None, 32, 32, 512)  2097664    ['activation_9[0][0]']           
 ose)                                                                                             
                                                                                                  
 concatenate (Concatenate)      (None, 32, 32, 1024  0           ['conv2d_transpose[0][0]',       
                                )                                 'activation_7[0][0]']           
                                                                                                  
 conv2d_10 (Conv2D)             (None, 32, 32, 512)  4719104     ['concatenate[0][0]']            
                                                                                                  
 batch_normalization_10 (BatchN  (None, 32, 32, 512)  2048       ['conv2d_10[0][0]']              
 ormalization)                                                                                    
                                                                                                  
 activation_10 (Activation)     (None, 32, 32, 512)  0           ['batch_normalization_10[0][0]'] 
                                                                                                  
 conv2d_11 (Conv2D)             (None, 32, 32, 512)  2359808     ['activation_10[0][0]']          
                                                                                                  
 batch_normalization_11 (BatchN  (None, 32, 32, 512)  2048       ['conv2d_11[0][0]']              
 ormalization)                                                                                    
                                                                                                  
 activation_11 (Activation)     (None, 32, 32, 512)  0           ['batch_normalization_11[0][0]'] 
                                                                                                  
 conv2d_transpose_1 (Conv2DTran  (None, 64, 64, 256)  524544     ['activation_11[0][0]']          
 spose)                                                                                           
                                                                                                  
 concatenate_1 (Concatenate)    (None, 64, 64, 512)  0           ['conv2d_transpose_1[0][0]',     
                                                                  'activation_5[0][0]']           
                                                                                                  
 conv2d_12 (Conv2D)             (None, 64, 64, 256)  1179904     ['concatenate_1[0][0]']          
                                                                                                  
 batch_normalization_12 (BatchN  (None, 64, 64, 256)  1024       ['conv2d_12[0][0]']              
 ormalization)                                                                                    
                                                                                                  
 activation_12 (Activation)     (None, 64, 64, 256)  0           ['batch_normalization_12[0][0]'] 
                                                                                                  
 conv2d_13 (Conv2D)             (None, 64, 64, 256)  590080      ['activation_12[0][0]']          
                                                                                                  
 batch_normalization_13 (BatchN  (None, 64, 64, 256)  1024       ['conv2d_13[0][0]']              
 ormalization)                                                                                    
                                                                                                  
 activation_13 (Activation)     (None, 64, 64, 256)  0           ['batch_normalization_13[0][0]'] 
                                                                                                  
 conv2d_transpose_2 (Conv2DTran  (None, 128, 128, 12  131200     ['activation_13[0][0]']          
 spose)                         8)                                                                
                                                                                                  
 concatenate_2 (Concatenate)    (None, 128, 128, 25  0           ['conv2d_transpose_2[0][0]',     
                                6)                                'activation_3[0][0]']           
                                                                                                  
 conv2d_14 (Conv2D)             (None, 128, 128, 12  295040      ['concatenate_2[0][0]']          
                                8)                                                                
                                                                                                  
 batch_normalization_14 (BatchN  (None, 128, 128, 12  512        ['conv2d_14[0][0]']              
 ormalization)                  8)                                                                
                                                                                                  
 activation_14 (Activation)     (None, 128, 128, 12  0           ['batch_normalization_14[0][0]'] 
                                8)                                                                
                                                                                                  
 conv2d_15 (Conv2D)             (None, 128, 128, 12  147584      ['activation_14[0][0]']          
                                8)                                                                
                                                                                                  
 batch_normalization_15 (BatchN  (None, 128, 128, 12  512        ['conv2d_15[0][0]']              
 ormalization)                  8)                                                                
                                                                                                  
 activation_15 (Activation)     (None, 128, 128, 12  0           ['batch_normalization_15[0][0]'] 
                                8)                                                                
                                                                                                  
 conv2d_transpose_3 (Conv2DTran  (None, 256, 256, 64  32832      ['activation_15[0][0]']          
 spose)                         )                                                                 
                                                                                                  
 concatenate_3 (Concatenate)    (None, 256, 256, 12  0           ['conv2d_transpose_3[0][0]',     
                                8)                                'activation_1[0][0]']           
                                                                                                  
 conv2d_16 (Conv2D)             (None, 256, 256, 64  73792       ['concatenate_3[0][0]']          
                                )                                                                 
                                                                                                  
 batch_normalization_16 (BatchN  (None, 256, 256, 64  256        ['conv2d_16[0][0]']              
 ormalization)                  )                                                                 
                                                                                                  
 activation_16 (Activation)     (None, 256, 256, 64  0           ['batch_normalization_16[0][0]'] 
                                )                                                                 
                                                                                                  
 conv2d_17 (Conv2D)             (None, 256, 256, 64  36928       ['activation_16[0][0]']          
                                )                                                                 
                                                                                                  
 batch_normalization_17 (BatchN  (None, 256, 256, 64  256        ['conv2d_17[0][0]']              
 ormalization)                  )                                                                 
                                                                                                  
 activation_17 (Activation)     (None, 256, 256, 64  0           ['batch_normalization_17[0][0]'] 
                                )                                                                 
                                                                                                  
 conv2d_18 (Conv2D)             (None, 256, 256, 8)  520         ['activation_17[0][0]']          
                                                                                                  
==================================================================================================
Total params: 31,055,752
Trainable params: 31,043,976
Non-trainable params: 11,776
__________________________________________________________________________________________________
In [ ]:
# Train model
print(f'Modèle : {model_name}')
model, hist_df, training_time = train_model(model, model_name, "categorical_crossentropy", augmented=False)
Modèle : U-Net_base
Récupération du modèle U-Net
In [ ]:
# Draw loss function and metric evolution
helpers.draw_history(hist_df)
In [ ]:
# Predict from model trained
y_pred_argmax = predict_model(model, model_name, "categorical_crossentropy", training_time, show_pred=True)
25/25 [==============================] - 449s 18s/step - loss: 1.0450 - mean_iou: 0.3021
Pour le meilleur modèle on obtient :
mean_iou : 0.30205994844436646
loss : 1.0450489521026611
0
1
2
3
4
5
6
7
8
9

Optimisation de la fonction loss¶

In [26]:
from tensorflow.python.keras.losses import categorical_crossentropy
from tensorflow.python.keras import backend as K

def dice_coeff(y_true, y_pred):
    smooth = 1.
    y_true_f = K.cast(K.flatten(y_true), K.floatx())
    y_pred_f = K.cast(K.flatten(y_pred), K.floatx())
    intersection = K.sum(y_true_f * y_pred_f)
    score = (2. * intersection + smooth) / (K.sum(y_true_f) + K.sum(y_pred_f) + smooth)
    return score

def dice_loss(y_true, y_pred):
    loss = 1 - dice_coeff(y_true, y_pred)
    return loss

def total_loss(y_true, y_pred):
    loss = categorical_crossentropy(y_true, y_pred) + (3*dice_loss(y_true, y_pred))
    return loss

Modèle Unet, fonction dice loss¶

  • Métrique Mean_iou
  • Fonction loss 'dice loss"
  • Pas d'augmentation de données
In [ ]:
# Build model
model = models.build_unet(INPUT_SHAPE, NB_CLASSES)
model_name = model.name + '_dice_loss'
softmax
In [ ]:
# Train model
print(f'Modèle : {model_name}')
model, hist_df, training_time = train_model(model, model_name, dice_loss, augmented=False)
Modèle : U-Net_dice_loss
Récupération du modèle U-Net
In [ ]:
# Draw loss function and metric evolution
helpers.draw_history(hist_df)
In [ ]:
# Predict from model trained
y_pred_argmax = predict_model(model, model_name, "dice_loss", training_time, show_pred=True)
25/25 [==============================] - 38s 1s/step - loss: 0.3764 - mean_iou: 0.2936
Pour le meilleur modèle on obtient :
mean_iou : 0.29356127977371216
loss : 0.37635427713394165
0
1
2
3
4
5
6
7
8
9

Modèle Unet, fonction combinaison loss¶

  • Métrique Mean_iou
  • Fonction loss 'combinaison loss" (dice loss & categorical crossentropy)
  • Pas d'augmentation de données
In [ ]:
# Build model
model = models.build_unet(INPUT_SHAPE, NB_CLASSES)
model_name = model.name + '_combine_loss'
softmax
In [ ]:
# Train model
print(f'Modèle : {model_name}')
model, hist_df, training_time = train_model(model, model_name, total_loss, augmented=False)
Modèle : U-Net_combine_loss
Récupération du modèle U-Net
In [ ]:
# Draw loss function and metric evolution
helpers.draw_history(hist_df)
In [ ]:
# Predict from model trained
y_pred_argmax = predict_model(model, model_name, "combine_loss", training_time, show_pred=True)
25/25 [==============================] - 38s 1s/step - loss: 2.4823 - mean_iou: 0.3026
Pour le meilleur modèle on obtient :
mean_iou : 0.3026266098022461
loss : 2.482293128967285
0
1
2
3
4
5
6
7
8
9

Modèle Unet data augmented¶

  • Métrique Mean_iou
  • Fonction loss 'combinaison loss"
  • Augmentation de données
In [ ]:
# Build model
model = models.build_unet(INPUT_SHAPE, NB_CLASSES)
model_name = model.name + '_augmented'
softmax
In [ ]:
# Train model
print(f'Modèle : {model_name}')
model, hist_df, training_time = train_model(model, model_name, total_loss, augmented=True)
Modèle : U-Net_augmented
Récupération du modèle U-Net
In [ ]:
# Draw loss function and metric evolution
helpers.draw_history(hist_df)
In [ ]:
# Predict from model trained
y_pred_argmax = predict_model(model, model_name, "combine_loss", training_time, show_pred=True)
25/25 [==============================] - 39s 2s/step - loss: 2.3302 - mean_iou: 0.3037
Pour le meilleur modèle on obtient :
mean_iou : 0.30368149280548096
loss : 2.330190658569336
0
1
2
3
4
5
6
7
8
9

Modèle Resnet50 Unet¶

  • Métrique Mean_iou
  • Fonction loss 'combinaison loss" (dice loss & categorical crossentropy)
  • Augmentation de données
In [ ]:
# Build model
model = models.build_resnet50_unet(INPUT_SHAPE, NB_CLASSES)
model_name = model.name + '_augmented'
Downloading data from https://storage.googleapis.com/tensorflow/keras-applications/resnet/resnet50_weights_tf_dim_ordering_tf_kernels_notop.h5
94773248/94765736 [==============================] - 1s 0us/step
94781440/94765736 [==============================] - 1s 0us/step
In [ ]:
# Train model
print(f'Modèle : {model_name}')
model, hist_df, training_time = train_model(model, model_name, total_loss, augmented=True)
Modèle : ResNet50_U-Net_augmented
Récupération du modèle ResNet50_U-Net
In [ ]:
# Draw loss function and metric evolution
helpers.draw_history(hist_df)
In [ ]:
# Predict from model trained
y_pred_argmax = predict_model(model, model_name, "combine_loss", training_time, show_pred=True)
25/25 [==============================] - 38s 1s/step - loss: 2.3114 - mean_iou: 0.3132
Pour le meilleur modèle on obtient :
mean_iou : 0.31320345401763916
loss : 2.311370849609375
WARNING:tensorflow:5 out of the last 5 calls to <function Model.make_predict_function.<locals>.predict_function at 0x7fb2f3e88c20> triggered tf.function retracing. Tracing is expensive and the excessive number of tracings could be due to (1) creating @tf.function repeatedly in a loop, (2) passing tensors with different shapes, (3) passing Python objects instead of tensors. For (1), please define your @tf.function outside of the loop. For (2), @tf.function has experimental_relax_shapes=True option that relaxes argument shapes that can avoid unnecessary retracing. For (3), please refer to https://www.tensorflow.org/guide/function#controlling_retracing and https://www.tensorflow.org/api_docs/python/tf/function for  more details.
0
1
2
3
4
5
6
7
8
9

Modèle Vgg16 Unet¶

  • Métrique Mean_iou
  • Fonction loss 'combinaison loss" (dice loss & categorical crossentropy)
  • Pas d'augmentation de données
In [33]:
# Build model
model = models.build_vgg16_unet(INPUT_SHAPE, NB_CLASSES)
model_name = model.name + '_base'
In [34]:
# Train model
print(f'Modèle : {model_name}')
model, hist_df, training_time = train_model(model, model_name, total_loss, augmented=False)
Modèle : VGG16_U-Net_base
Récupération du modèle VGG16_U-Net
In [35]:
# Draw loss function and metric evolution
helpers.draw_history(hist_df)
In [36]:
# Predict from model trained
y_pred_argmax = predict_model(model, model_name, "combine_loss", training_time, show_pred=True)
25/25 [==============================] - 33s 1s/step - loss: 2.3707 - mean_iou: 0.3053
Pour le meilleur modèle on obtient :
mean_iou : 0.30532243847846985
loss : 2.3706603050231934
0
1
2
3
4
5
6
7
8
9

Modèle Vgg16 Unet, augmented¶

  • Métrique Mean_iou
  • Fonction loss 'combinaison loss" (dice loss & categorical crossentropy)
  • Augmentation de données
In [24]:
# Build model
model = models.build_vgg16_unet(INPUT_SHAPE, NB_CLASSES)
model_name = model.name + '_augmented'
Downloading data from https://storage.googleapis.com/tensorflow/keras-applications/vgg16/vgg16_weights_tf_dim_ordering_tf_kernels_notop.h5
58892288/58889256 [==============================] - 0s 0us/step
58900480/58889256 [==============================] - 0s 0us/step
In [27]:
# Train model
print(f'Modèle : {model_name}')
model, hist_df, training_time = train_model(model, model_name, total_loss, augmented=True)
Modèle : VGG16_U-Net_augmented
Epoch 1/50
297/297 [==============================] - 1606s 5s/step - loss: 3.4115 - mean_iou: 0.1375 - val_loss: 3.3118 - val_mean_iou: 0.1576
Epoch 2/50
297/297 [==============================] - 253s 849ms/step - loss: 2.7443 - mean_iou: 0.1908 - val_loss: 2.8461 - val_mean_iou: 0.1831
Epoch 3/50
297/297 [==============================] - 250s 839ms/step - loss: 2.6220 - mean_iou: 0.2095 - val_loss: 2.7006 - val_mean_iou: 0.1903
Epoch 4/50
297/297 [==============================] - 252s 846ms/step - loss: 2.5556 - mean_iou: 0.2163 - val_loss: 2.6380 - val_mean_iou: 0.2051
Epoch 5/50
297/297 [==============================] - 253s 843ms/step - loss: 2.5222 - mean_iou: 0.2244 - val_loss: 2.6038 - val_mean_iou: 0.2141
Epoch 6/50
297/297 [==============================] - 249s 834ms/step - loss: 2.4930 - mean_iou: 0.2300 - val_loss: 2.5923 - val_mean_iou: 0.2114
Epoch 7/50
297/297 [==============================] - 250s 838ms/step - loss: 2.4847 - mean_iou: 0.2318 - val_loss: 2.5554 - val_mean_iou: 0.2223
Epoch 8/50
297/297 [==============================] - 252s 843ms/step - loss: 2.4647 - mean_iou: 0.2371 - val_loss: 2.5528 - val_mean_iou: 0.2244
Epoch 9/50
297/297 [==============================] - 248s 830ms/step - loss: 2.4517 - mean_iou: 0.2423 - val_loss: 2.5291 - val_mean_iou: 0.2376
Epoch 10/50
297/297 [==============================] - 250s 836ms/step - loss: 2.4374 - mean_iou: 0.2481 - val_loss: 2.5417 - val_mean_iou: 0.2328
Epoch 11/50
297/297 [==============================] - 251s 840ms/step - loss: 2.4219 - mean_iou: 0.2517 - val_loss: 2.5477 - val_mean_iou: 0.2330
Epoch 12/50
297/297 [==============================] - 255s 850ms/step - loss: 2.4012 - mean_iou: 0.2551 - val_loss: 2.5222 - val_mean_iou: 0.2388
Epoch 13/50
297/297 [==============================] - 253s 843ms/step - loss: 2.3894 - mean_iou: 0.2573 - val_loss: 2.5285 - val_mean_iou: 0.2419
Epoch 14/50
297/297 [==============================] - 249s 835ms/step - loss: 2.3731 - mean_iou: 0.2593 - val_loss: 2.5134 - val_mean_iou: 0.2345
Epoch 15/50
297/297 [==============================] - 250s 839ms/step - loss: 2.3663 - mean_iou: 0.2616 - val_loss: 2.4945 - val_mean_iou: 0.2466
Epoch 16/50
297/297 [==============================] - 250s 837ms/step - loss: 2.3523 - mean_iou: 0.2639 - val_loss: 2.4783 - val_mean_iou: 0.2472
Epoch 17/50
297/297 [==============================] - 250s 836ms/step - loss: 2.3376 - mean_iou: 0.2667 - val_loss: 2.4736 - val_mean_iou: 0.2472
Epoch 18/50
297/297 [==============================] - 249s 833ms/step - loss: 2.3298 - mean_iou: 0.2686 - val_loss: 2.4727 - val_mean_iou: 0.2442
Epoch 19/50
297/297 [==============================] - 247s 828ms/step - loss: 2.3251 - mean_iou: 0.2687 - val_loss: 2.4665 - val_mean_iou: 0.2463
Epoch 20/50
297/297 [==============================] - 248s 829ms/step - loss: 2.3141 - mean_iou: 0.2706 - val_loss: 2.4778 - val_mean_iou: 0.2415
Epoch 21/50
297/297 [==============================] - 251s 840ms/step - loss: 2.3046 - mean_iou: 0.2731 - val_loss: 2.4792 - val_mean_iou: 0.2474
Epoch 22/50
297/297 [==============================] - 252s 842ms/step - loss: 2.2976 - mean_iou: 0.2740 - val_loss: 2.4628 - val_mean_iou: 0.2498
Epoch 23/50
297/297 [==============================] - 245s 821ms/step - loss: 2.2872 - mean_iou: 0.2762 - val_loss: 2.4822 - val_mean_iou: 0.2431
Epoch 24/50
297/297 [==============================] - 248s 832ms/step - loss: 2.2833 - mean_iou: 0.2757 - val_loss: 2.4635 - val_mean_iou: 0.2468
Epoch 25/50
297/297 [==============================] - 249s 834ms/step - loss: 2.2744 - mean_iou: 0.2777 - val_loss: 2.4555 - val_mean_iou: 0.2486
Epoch 26/50
297/297 [==============================] - 248s 831ms/step - loss: 2.2680 - mean_iou: 0.2789 - val_loss: 2.4813 - val_mean_iou: 0.2397
Epoch 27/50
297/297 [==============================] - 250s 838ms/step - loss: 2.2611 - mean_iou: 0.2806 - val_loss: 2.4770 - val_mean_iou: 0.2358
Epoch 28/50
297/297 [==============================] - 250s 837ms/step - loss: 2.2592 - mean_iou: 0.2813 - val_loss: 2.4702 - val_mean_iou: 0.2414
Epoch 29/50
297/297 [==============================] - 247s 828ms/step - loss: 2.2485 - mean_iou: 0.2831 - val_loss: 2.4987 - val_mean_iou: 0.2400
Epoch 30/50
297/297 [==============================] - 249s 835ms/step - loss: 2.2441 - mean_iou: 0.2849 - val_loss: 2.4595 - val_mean_iou: 0.2436
Epoch 31/50
297/297 [==============================] - 249s 835ms/step - loss: 2.2363 - mean_iou: 0.2863 - val_loss: 2.4592 - val_mean_iou: 0.2454
Epoch 32/50
297/297 [==============================] - 249s 835ms/step - loss: 2.2319 - mean_iou: 0.2865 - val_loss: 2.4548 - val_mean_iou: 0.2480
Epoch 33/50
297/297 [==============================] - 245s 821ms/step - loss: 2.2259 - mean_iou: 0.2887 - val_loss: 2.4571 - val_mean_iou: 0.2464
Epoch 34/50
297/297 [==============================] - 248s 830ms/step - loss: 2.2207 - mean_iou: 0.2892 - val_loss: 2.4473 - val_mean_iou: 0.2511
Epoch 35/50
297/297 [==============================] - 246s 825ms/step - loss: 2.2179 - mean_iou: 0.2899 - val_loss: 2.4675 - val_mean_iou: 0.2459
Epoch 36/50
297/297 [==============================] - 244s 816ms/step - loss: 2.2182 - mean_iou: 0.2898 - val_loss: 2.4751 - val_mean_iou: 0.2420
Epoch 37/50
297/297 [==============================] - 249s 835ms/step - loss: 2.2073 - mean_iou: 0.2922 - val_loss: 2.4582 - val_mean_iou: 0.2446
Epoch 38/50
297/297 [==============================] - 251s 841ms/step - loss: 2.2025 - mean_iou: 0.2932 - val_loss: 2.4918 - val_mean_iou: 0.2411
Epoch 39/50
297/297 [==============================] - 246s 824ms/step - loss: 2.1997 - mean_iou: 0.2937 - val_loss: 2.4809 - val_mean_iou: 0.2415
Epoch 40/50
297/297 [==============================] - 245s 820ms/step - loss: 2.1951 - mean_iou: 0.2947 - val_loss: 2.4631 - val_mean_iou: 0.2501
Epoch 41/50
297/297 [==============================] - 249s 833ms/step - loss: 2.1928 - mean_iou: 0.2955 - val_loss: 2.4816 - val_mean_iou: 0.2452
Epoch 42/50
297/297 [==============================] - 248s 830ms/step - loss: 2.1891 - mean_iou: 0.2955 - val_loss: 2.5053 - val_mean_iou: 0.2375
Epoch 43/50
297/297 [==============================] - 247s 827ms/step - loss: 2.1896 - mean_iou: 0.2958 - val_loss: 2.4851 - val_mean_iou: 0.2468
Epoch 44/50
297/297 [==============================] - 245s 822ms/step - loss: 2.1799 - mean_iou: 0.2972 - val_loss: 2.4719 - val_mean_iou: 0.2479
Epoch 45/50
297/297 [==============================] - 247s 828ms/step - loss: 2.1797 - mean_iou: 0.2975 - val_loss: 2.4740 - val_mean_iou: 0.2462
Epoch 46/50
297/297 [==============================] - 247s 827ms/step - loss: 2.1724 - mean_iou: 0.2996 - val_loss: 2.4587 - val_mean_iou: 0.2464
Epoch 47/50
297/297 [==============================] - 247s 827ms/step - loss: 2.1725 - mean_iou: 0.2992 - val_loss: 2.5075 - val_mean_iou: 0.2391
Epoch 48/50
297/297 [==============================] - 247s 826ms/step - loss: 2.1652 - mean_iou: 0.3016 - val_loss: 2.4816 - val_mean_iou: 0.2453
Epoch 00048: early stopping
INFO:tensorflow:Assets written to: saved/models/VGG16_U-Net_augmented/assets
In [28]:
# Draw loss function and metric evolution
helpers.draw_history(hist_df)
In [29]:
# Predict from model trained
y_pred_argmax = predict_model(model, model_name, "combine_loss", training_time, show_pred=True)
25/25 [==============================] - 33s 1s/step - loss: 2.2817 - mean_iou: 0.3168
Pour le meilleur modèle on obtient :
mean_iou : 0.31677231192588806
loss : 2.2816824913024902
0
1
2
3
4
5
6
7
8
9

Modèle Vgg16 Unet, avec freeze¶

  • Métrique Mean_iou
  • Fonction loss 'combinaison loss" (dice loss & categorical crossentropy)
  • Freeze des couches
In [ ]:
# Build model
model = models.build_vgg16_unet(INPUT_SHAPE, NB_CLASSES)
model_name = model.name + '_freeze'
models.freeze_layer_vgg16(model)
freeze layer: block1_conv1
freeze layer: block1_conv2
freeze layer: block1_pool
freeze layer: block2_conv1
freeze layer: block2_conv2
freeze layer: block2_pool
freeze layer: block3_conv1
freeze layer: block3_conv2
freeze layer: block3_conv3
freeze layer: block3_pool
freeze layer: block4_conv1
freeze layer: block4_conv2
freeze layer: block4_conv3
freeze layer: block4_pool
In [ ]:
# Train model
print(f'Modèle : {model_name}')
model, hist_df, training_time = train_model(model, model_name, total_loss, augmented=False)
In [ ]:
# Draw loss function and metric evolution
helpers.draw_history(hist_df)
In [ ]:
# Predict from model trained
y_pred_argmax = predict_model(model, model_name, "combine_loss", training_time, show_pred=True)
25/25 [==============================] - 38s 1s/step - loss: 2.2608 - mean_iou: 0.3193
Pour le meilleur modèle on obtient :
mean_iou : 0.3192502558231354
loss : 2.260753870010376
0
1
2
3
4
5
6
7
8
9

Synthèse comparative¶

In [ ]:
pd.DataFrame.from_dict(synthese, orient='index')
Out[ ]:
loss type loss score mean_iou Training time Predict time
U-Net_base categorical_crossentropy 1.045049 0.302060 6005.426807 1.509681
U-Net_dice_loss dice_loss 0.376354 0.293561 11075.384537 0.595256
U-Net_combine_loss combine_loss 2.482293 0.302627 8733.732015 0.608075
U-Net_augmented combine_loss 2.330191 0.303681 9423.844568 0.646507
ResNet50_U-Net_augmented combine_loss 2.311371 0.313203 12474.215919 1.753642
VGG16_U-Net_base combine_loss 2.267051 0.325019 8658.989292 2.976111
VGG16_U-Net_augmented combine_loss 2.298635 0.319571 10783.304042 0.527344
VGG16_U-Net_freeze combine_loss 2.260754 0.319250 9002.626294 0.582971